use zlib-rs for gzip compression in rust code
#15417
Conversation
Various C dependencies (curl, git) still rely on `libz-sys`, so for the time being a system libc is still required. But, using zlib-rs via flate2 is straightforward, and gives good speedup for `cargo package`. It is also extremely portable, because it's just rust code.
rustbot
added
A-documenting-cargo-itself
joshtriplett
added
I-nominated-to-discuss
|
Thanks, @folkertdev! Nominating for next week's Cargo meeting. |
ehuss
removed
the
I-nominated-to-discuss
|
We talked about how we want to approach these kind of changes in today's Cargo team meeting. For gitoxide, its still under active development and each feature needs a lot of vetting before adoption, so we're taking an incremental, feature-at-at-ime, approach with an opt-in and opt-out windows to ensure as smooth of a path as possible. For something like zlib-rs, it feels smaller and more mature. We likely don't need a a multi-release transition window. We do want it merged at the beginning of a release window to give as much runtime within the release as possible. We also appreciate how easy it would be to back this out during beta if need be. The main risk we see with zlib-rs is platform support. For our As this is still early in the release Window, we can move forward with this. |
Update cargo 4 commits in 864f74d4eadcaea3eeda37a2e7f4d34de233d51e..d811228b14ae2707323f37346aee3f4147e247e6 2025-04-11 20:37:27 +0000 to 2025-04-15 15:18:42 +0000 - use `zlib-rs` for gzip compression in rust code (rust-lang/cargo#15417) - test(rustfix): Use `snapbox` for snapshot testing (rust-lang/cargo#15429) - chore(deps): update rust crate gix to 0.71.0 [security] (rust-lang/cargo#15391) - Make sure search paths inside OUT_DIR precede external paths (rust-lang/cargo#15221) Also, * The license exception of sha1_smol with BSD-3-Clause is no longer needed, as `gix-*` doesn't depend on it. * Cargo depends on zlib-rs, which is distributed under Zlib license r? ghost
Update cargo 4 commits in 864f74d4eadcaea3eeda37a2e7f4d34de233d51e..d811228b14ae2707323f37346aee3f4147e247e6 2025-04-11 20:37:27 +0000 to 2025-04-15 15:18:42 +0000 - use `zlib-rs` for gzip compression in rust code (rust-lang/cargo#15417) - test(rustfix): Use `snapbox` for snapshot testing (rust-lang/cargo#15429) - chore(deps): update rust crate gix to 0.71.0 [security] (rust-lang/cargo#15391) - Make sure search paths inside OUT_DIR precede external paths (rust-lang/cargo#15221) Also, * The license exception of sha1_smol with BSD-3-Clause is no longer needed, as `gix-*` doesn't depend on it. * Cargo depends on zlib-rs, which is distributed under Zlib license r? ghost
What does this PR try to resolve?
This PR uses
zlib-rsvia theflate2crate. It is used for (de)compressing gzip files (e.g. incargo package).Using zlib-rs is significantly faster, and produces output of roughly similar size. For the
windows-bindgencrate(a very large crate), the speedup is over 60%, or about 2 seconds.
How should we test and review this PR?
Generally CI/the test suite should handle correctness.
Something to look out for is zlib-rs producing larger binaries than before. So far we're seeing output sizes that are roughly the same (sometimes a bit better, sometimes a bit worse) as the status quo.
We've not specifically looked at decompression yet, mostly because we could not come up with a good command to benchmark. In general zlib-rs is much faster than stock zlib there too.
Additional information
For the time being,
cargostill depends on stock zlib via e.g.curlandgit. As far as I know it is the intention to eventually move away from these C dependencies, at which point the dependency on stock zlib also disappears.